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1. Summary . A stochastic approximation algorithm for estimating the proportions 
in a mixture of normal densities is presented. The algorithm is sho™ to con- 
verge to the true proportions in the case of a mixture of two normal densities. 

2, Introduction. Let A = {a <• R"*:a. ? 0 and ^4^ =1}. For each 1. 

1 1=1 1 ’ 

i = l,...,ra, let be an element of R*' and be a positive definite 

real symmetric n x n matrix. Let X be a random variable with values in 
and with density function. 

p(a,x) = a^p^(x), for x e r" 

where a f A and 

p^(x) = (211) ^^^exp{- y(x-p^)^E^“^(x-P^)} 

for each i = l,...,m. 

We assume that a is not known but that and are known for 

i = l,...,m. An algorithm for estimating S will be presented in part 3 of 
this paper and in part 4 the algorithm xstlll be shown to converge to a in mean 
square and with probability 1 in the case where m = 2. 
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3. Tho AlKorllhm . Let I’C « .eciuence of observations on X„ Let 

n+l 

a° e A. For n Ji 0 define, ct by 


n+1 

^1 


ct., 


c (a” 
n 1 




■). 


where 


m ^ 

P ii(x ) = .E.ct .p, (x ) 
* a n 1==1 i 1 n 


and 


K ”0 ^ sequence of positive numbers such tliat 


00 

,2 c, = “ and 
k=o k 


2 


OO 

1 ^ c, 

k=o k 


< ® 


We note that each iterate is in A and that, since X is a random variable, 
each iterate may itself be considered a random variable. 

4 . ConverRencP of the Algorithm . 

Theorem ; If ct e then the algorithm described in part 3 converges to “S 
in mean square and with probability 1. 


Proof : We refer the reader to the algorithm described in [l>pp. 332-333] and 

to the proof of convergence given in [l,pp. 350-352]. Tlie applicability of the 
theorem given there is clear if we let f(a) = E(Z^), for each a e A, where 

ct,, (p . o X) 

\ _ J. J, 
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In order to show convergence we must show that conditions (A1)-(A3) in 
fljpp. 332-333] are satisfied. First we note chat 


f(ct) = - 0202 (a2)) 


where 




h 

r" 


Pl<x) 


aj^Pj^(x) 


(l-cXj^)P2(x) 


P^(x)dx 


and 





("l-a^Pj^"(x) + a2P2<x) 


p^(x)dx. 


Further, we note that 


d f Pi(x)rPj^(x) - P2(x)]^ 

y = I T ' PjN(x)dx > 0 

daj + (l-a^)p2(x)]'^ “ 

R 


and 


d 82(^2) r p2(x)CP2W " Pj^Cx)]^ 

5 “ / ^*pA(x)dx > 0 . 

d»2 J [(l-a2)Pj^(x) +cx2P2(x)]'^ “ 
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Now, BiCa,) •= 1 nnd g, (1) = 1. So, since g. has positive second derivative 
we have that < i If ^ ^ ^ e (0,0^^). 

Similarly, ° ^ 82^) = 1 63 (“2^ ^ ^ “2 ^ 

and 82(^2^ ^ ^ “2 

We now show that (A1)“(AJ) are satisfied; het ct e A. Then 

(Al) f(a) * 0 iff 8i<aj) = 1 = 82(^2^ « = a. 

(A 2 ) (a-a)’-'^f(a) = (a^-ot^^) (a^-a^gj^(aj)) + (a 2 -a 2 ) (02 ““ 2 ® 2 ^°‘ 2 ^ ^* 

If Cj^ > then ^ ^lod > 0- Then also 

< C?2 and g2^“2^ ^ ^ ^ 

ot^ > etj^ then (a-a)*^f(a) >0. Similarly, if ^ then 
/\ T 

(a~a) f(a) > 0. Thus, A2 is satisfied in any closed, convex 
subset of A. 


(A3) 


R(iizjr) 


3 2 

i^i <«i 


- 2 


f 4 ^l 

J Pa 


(x) 


p,^(x)dx + 


■1^. 

r" 


(x) 


pA(x)dx) 


Now, we note that each term in the ith summand, i - 1,2, is 
.less than 1 so that there is an h > 0 such that E(||z^|| ) < h 
for all a e A and A3 is satisfied. 
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